Robot system

ABSTRACT

A robot system including a conveying device; a robot that processes an object being conveyed; a first and second supply unit that supply the object onto the conveying device; a movement amount detection unit that successively detects an amount of movement of the object supplied by the first supply unit; a single vision sensor that successively acquires visual information of the object being conveyed; an object detection unit that processes the acquired visual information to detect position and orientation of the object; an interval detection unit that detects an interval between objects on the conveying device; a control unit that controls the robot based on the amount of movement and the position and orientation; and a production management unit that causes the second supply unit to supply the object at a position of the interval.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Patent Application No.2017-154042, the content of which is incorporated herein by reference.

FIELD

The present invention relates to a robot system.

BACKGROUND

Conventionally, there is known a robot system for picking up, from aconveyor, an object being conveyed by the conveyor, by detectingposition and orientation of the object and tracking the object by arobot (for example, see PTL 1).

The robot system in PTL 1 includes a temporary placing table where anobject picked up from the conveyor is temporarily stored in a case wherethere is an excessive supply of objects by the conveyor, and an objectstored on the temporary placing table is picked up to be used whensupply of objects by the conveyor becomes insufficient.

PTL 1 Japanese Unexamined Patent Application, Publication No.2012-188231 SUMMARY

An aspect of the present invention provides a robot system including aconveying device that conveys an object in one direction; a robot,installed near the conveying device, that performs a process on theobject being conveyed by the conveying device; a first supply unit and asecond supply unit that supply the object onto the conveying device; amovement amount detection unit that successively detects an amount ofmovement, by the conveying device, of the object supplied by the firstsupply unit onto the conveying device; a single vision sensor thatsuccessively acquires, on an upstream side of the robot in a conveyingdirection, visual information of the object being conveyed by theconveying device; an object detection unit that processes the visualinformation acquired by the vision sensor to detect position andorientation of the object; an interval detection unit that processes thevisual information acquired by the vision sensor to detect an intervalbetween objects on the conveying device in the conveying direction; acontrol unit that controls the robot based on the amount of movement ofthe object detected by the movement amount detection unit and theposition and orientation of the object detected by the object detectionunit; and a production management unit that causes the second supplyunit to supply the object at a position of the interval detected by theinterval detection unit, in a case where the interval is greater than apredetermined threshold.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is an overall configuration diagram showing a robot systemaccording to an embodiment of the present invention.

FIG. 2 is a block diagram showing a control unit provided in the robotsystem in FIG. 1.

FIG. 3 is a plan view showing, with respect to the robot system in FIG.1, a relationship among a conveyor, a field of view of a camera, and anobject presence check region.

FIG. 4 is a diagram describing tracking coordinate transformationaccording to the robot system in FIG. 1.

FIG. 5 is a partial side view showing an example of when absence of anobject in the object presence check region is detected by the robotsystem in FIG. 1.

FIG. 6 is a partial side view showing an example of when an object issupplied from a second supply unit after the detection in FIG. 5.

FIG. 7 is a partial side view showing an example of when an image of theobject supplied in FIG. 6 is captured.

FIG. 8 is an overall configuration diagram showing a modification of therobot system in FIG. 1.

FIG. 9 is a diagram showing another modification of the robot system inFIG. 1, describing a case of detecting velocity of movement of an objectby a conveyor from acquired images.

DETAILED DESCRIPTION

Hereinafter, a robot system 1 according to an embodiment of the presentinvention will be described with reference to the drawings.

As shown in FIG. 1, the robot system 1 according to the presentembodiment includes a conveyor (conveying device) 2 that conveys objectsO, a first supply unit 6 that supplies the objects O onto the conveyor 2at an upstream side of the conveyor 2, a robot 3 installed near theconveyor 2, a camera (vision sensor) 4 installed facing downward abovethe conveyor 2, on the upstream side of the robot 3 in a conveyingdirection, a control unit 5 that controls the robot 3 based on an image(visual information) acquired by the camera 4, and a second supply unit7 that supplies the objects O into a field of view of the camera 4.

For example, the conveyor 2 is a belt conveyor, and includes a belt 8that convey, in one direction, the objects O which are placed on thebelt 8. The belt 8 is driven by a motor 8 a. The motor 8 a is providedwith an encoder (movement amount detection unit) 9 that detects arotation angle of the motor 8 a, and indirectly detects the amount ofmovement of the object O by the belt 8 based on a reduction ratio of adrive force transmission mechanism (not shown) from the motor 8 a to thebelt 8. Furthermore, the encoder 9 does not necessarily have to beprovided at the motor 8 a, and a rotary disc may be attached to arotation shaft of the encoder 9 and the rotary disc may be pressedagainst the belt 8.

The robot 3 may be of any type such as a horizontal mounting type or aceiling mounting type, and a robot hand 10 which is capable of grippingthe object O is provided at a distal end of a wrist of the robot 3, forexample.

The camera 4 has a field of view (see FIG. 3) A which is fixed at apartial region of the conveyor 2 in the conveying direction, andacquires a two-dimensional image of the object O being conveyed on theconveyor 2. A frame rate of the camera 4 is set in advance in such a waythat image-capturing is performed at least twice while the same object Ois passing through the field of view A.

As shown in FIG. 2, the control unit 5 includes an image processing unit11 that processes an image acquired by the camera 4 to recognize theobject O being conveyed by the conveyor 2, an object detection unit 12that detects position and orientation of the object O recognized by theimage processing unit 11, a space detection unit (interval detectionunit) 13 that detects an empty space (interval) between the recognizedobjects O, a drive control unit (control unit) 14 that controls therobot 3 based on the position and orientation of the object O detectedby the object detection unit 12 and the amount of movement of the objectO detected by the encoder 9, and a production management unit 15 thatcauses the object O to be supplied by the second supply unit 7, in acase where the space detected by the space detection unit 13 is greaterthan a predetermined size (threshold). The control unit 5 is configuredby hardware such as the CPU, memory and the like, which are not shown.

The drive control unit 14 causes the robot 3 to operate according to anoperation program which is taught in advance, and also, performstracking of following the object O on the conveyor 2 based on the amountof movement of the object O detected by the encoder 9, and causes therobot hand 10 attached to the distal end of the wrist to perform aprocess of gripping the moving object O and picking up the object O fromthe conveyor 2.

More specifically, the image processing unit 11 is to recognize anobject O in an acquired image by a technique such as pattern matching,for example. Furthermore, the object detection unit 12 detectspresence/absence of the object O in the image of the object O recognizedby the image processing unit 11, and in the case of presence of theobject O, the object detection unit 12 detects two-dimensionalcoordinates (position) and a rotation angle (orientation) around avertical axis of the object O.

Furthermore, as shown in FIG. 3, the space detection unit 13 sets acertain object presence check region B in the field of view A of thecamera 4, and detects whether or not an object O is present in theobject presence check region B. Moreover, in the case where an object Ois not present in the object presence check region B, the spacedetection unit 13 determines that there is enough empty space forputting in an object O, and outputs a signal to the effect to theproduction management unit 15.

The object presence check region B set in the field of view A of thecamera 4 is arranged on the upstream side with a space allowingacquisition of an image of an object O by the camera 4, from adownstream end of the field of view A of the camera 4. In the presentembodiment, the field of view A of the camera 4 is equally divided intoan upstream side and a downstream side in the conveying direction of theconveyor 2, and the object presence check region B is set on theupstream side.

For example, the first supply unit 6 is a transfer device thattransfers, at predetermined intervals, objects O conveyed from anotherconveyor, not shown, to the conveyor 2 of the robot system 1 accordingto the present embodiment. For example, the objects O are conveyed atrandom intervals by the other conveyor. Conveying velocity of theobjects O by the conveyor 2 of the robot system 1 according to thepresent embodiment is maintained constant at all times, and thus, whenthe objects O are conveyed by the other conveyor in a closely packedmanner, the other conveyor is decelerated, for example, to allowtransfer to the conveyor 2 of the robot system 1 according to thepresent embodiment to be performed at equal intervals. On the otherhand, if the objects O are conveyed by the other conveyor in a sparsemanner, transfer is performed in a thinned manner, and intervals betweenthe objects O on the conveyor 2 are increased.

The second supply unit 7 is a hopper which contains a large number ofobjects O, and which is capable of supplying the objects O one by oneaccording to a supply command signal from the production management unit15.

As shown in FIG. 3, the second supply unit 7 is arranged to supplyobjects O on the downstream side, in the conveying direction, of acenter position of the object presence check region B by a predetermineddistance L. The predetermined distance L is set to a value which isobtained by multiplying together time required for the object O to besupplied onto the conveyor 2 after a command signal from the productionmanagement unit 15 is received and standard velocity of the conveyor 2.

An operation of the robot system 1 according to the present embodimentconfigured in the above manner will be described below.

With the robot system 1 according to the present embodiment, when anobject O is supplied by the first supply unit 6 onto the conveyor 2, thesupplied object O is conveyed in one direction by the conveyor 2, and animage of the object O is captured by the camera 4 while the object O ispassing through the range of the field of view A of the camera 4. Animage acquired by the image-capturing is sent to the image processingunit 11 to be subjected to image processing, and the object O is therebyrecognized.

Then, information about the recognized object O is sent to the objectdetection unit 12 and the space detection unit 13, and the position andorientation of the object O are detected by the object detection unit12, and also, whether or not at least a predetermined interval ispresent between the objects O is detected by the space detection unit13. For example, if no object O is detected, the interval between theobjects O is equivalent to at least a width of the object presence checkregion B in a flow direction of the conveyor 2, and it is determinedthat at least a predetermined interval is present.

As shown in FIG. 4, the drive control unit 14 sets a tracking coordinatesystem TF based on the position and orientation of the object O detectedby the object detection unit 12, and calculates the amount of movementof the object O by

(e2−e1)/Scale,

where e1, e2 are encoder counts detected by the encoder 9 at animage-capturing time point and a current time point, respectively, andScale is a constant indicating a relationship between the encoder countse1, e2 and the amount of movement of the conveyor 2. Then, a currenttracking coordinate system TF′ is calculated by multiplying a coordinatetransformation matrix T which takes the amount of movement as acomponent by the tracking coordinate system TF.

TF′=T·TF

Furthermore, in FIG. 4, a reference sign a indicates the position andorientation of the object O seen from the coordinate system TF at theimage-capturing time point, and a reference sign a′ indicates theposition and orientation of the object O currently seen from thecoordinate system TF′. Reference signs X, Y, X′, Y′ indicate coordinateaxes of the coordinate systems TF, TF′.

The drive control unit 14 can move the robot hand 10 to follow theobject O being conveyed by the conveyor 2, with reference to thecalculated tracking coordinate system TF′, and can grip and pick up theobject O from the conveyor 2.

As shown in FIG. 5, with the robot system 1 according to the presentembodiment, in the case where absence of the object O in the objectpresence check region B set in the field of view A of the camera 4 isdetected by the object detection unit 12 by processing an image acquiredby the camera 4 at a certain time point, a supply command is output fromthe production management unit 15. The second supply unit 7 is caused tooperate by the supply command, and the object O is supplied by thesecond supply unit 7 onto the conveyor 2. In the drawing, a black circleindicates the position of the conveyor 2 corresponding to the centerposition of the object presence check region B at the time ofimage-capturing.

That is, in the case where there is no object O in the object presencecheck region B, an empty space of at least the range of the objectpresence check region B is present, and thus, by supplying the object Oin the empty space, an advantage that reduction in productivity of therobot 3 may be prevented by solving deficiency in the supply of objectsO can be achieved.

In this case, as shown in FIG. 6, because the hopper configuring thesecond supply unit 7 is arranged on the downstream side of the center ofthe object presence check region B by the predetermined distance L, evenif the conveyor 2 moves by a distance L corresponding to a time lag tLfrom output of the supply command from the production management unit 15at the time point when absence of the object O is confirmed to supply ofthe object O, the object (shaded object) O can be accurately supplied atthe position of the empty space, or at the position of the black circle,which is arranged at the center of the object presence check region B atthe time of image-capturing, and an advantage that a gap to thepreceding object O or the following object O is not made too small canbe achieved.

Furthermore, in the present embodiment, because the field of view A ofthe camera 4 is equally divided into two in the conveying direction, andthe object presence check region B is arranged in the region on theupstream side, an object O which is supplied by the second supply unit 7due to detection of absence of an object O in the object presence checkregion B at first image-capturing is then captured by secondimage-capturing at the time of passing through the region on thedownstream side of the field of view A, as shown in FIG. 7, and thus,the position and orientation of the object O can be detected.

That is, with the robot system 1 according to the present embodiment,presence/absence of an empty space on the conveyor 2 is detected byusing an image which is acquired by the camera 4 by capturing an objectO being conveyed by the conveyor 2 to detect the position andorientation of the object O, and an object O is supplied in the emptyspace, and thus, there are advantages that a new vision sensor does nothave to be added, that a temporary placing table is not necessary andthe robot 3 does not have to be unnecessarily moved, and also, thatdeficiency in supply of objects O by the first supply unit 6 can besupplemented and productivity can be increased.

Furthermore, there is an advantage that the robot 3 does not have todetermine movement of an object O to a temporary placing table orpicking up of an object O from the temporary placing table, and theoperation program is thus not complicated.

Additionally, the present embodiment describes the robot system 1provided with a single robot 3 and a single control unit 5, but instead,as shown in FIG. 8, a plurality of robots 3 may be arranged along theconveying direction of the conveyor 2, and respective control units 5may be connected to an upper-level cell control device 16.

In the case where a plurality of robots 3 are to perform tasks onobjects O conveyed by the same conveyor 2, the positions andorientations of objects O calculated based on an image captured by onecamera 4 may be used for processing by all the robots 3, and objects Omay be supplied according to spaces between objects O.

In this case, even if power of one of the robots 3 is switched off formaintenance or the like, a production management unit 15 provided in thecell control device 16 may output a supply command to the second supplyunit 7 so that the quantity of objects O conveyed by the conveyor 2 isappropriately adjusted in accordance with the processing amount of therest of the robots 3.

Furthermore, in the present embodiment, the amount of movement of theobject O by the conveyor 2 is detected by using the encoder 9, butinstead, the amount of movement of the object O may be detected by usingan image which is acquired by the camera 4 that detects the position andorientation of the object O and that detects spaces between objects O.

In this case, as shown in FIG. 9, objects O in a plurality of images ofthe same field of view acquired at different time points t1, t2, t3 at apredetermined time interval Δt are recognized, and coordinate positionsd1, d2, d3 of centers of gravity of the recognized objects O arecalculated.

Then, objects O, the centers of gravity of which are near a samecoordinate in a direction orthogonal to the conveying direction inimages acquired adjacently in a time axis direction, may be recognizedto be the same object O, and conveying velocity V may be calculated bydividing each of differences d3−d2, d2−d1 in coordinate values of thecenters of gravity of the object O in the conveying direction by thetime interval Δt of image-capturing. In the case where the conveyingvelocity V is calculated several times for the same object O, an averagevalue or a value obtained by performing fitting by least squares methodor the like may be output as the conveying velocity.

Moreover, in this case, a frame rate of images for checkingpresence/absence of an object O in the object presence check region Band for checking the position/orientation of the object O may be setlow, and a frame rate of images for detecting the conveying velocity Vof an object O by the conveyor 2 may be set high. By detecting theconveying velocity V of an object O from images acquired at a high framerate, the detection accuracy may be increased.

Moreover, in the present embodiment, the frame rate of the camera 4 isset in advance, but instead, the frame rate may be changed according tothe amount of movement of the object O detected by the encoder 9.

From the above-described embodiment, the following invention is derived.

An aspect of the present invention provides a robot system including aconveying device that conveys an object in one direction; a robot,installed near the conveying device, that performs a process on theobject being conveyed by the conveying device; a first supply unit and asecond supply unit that supply the object onto the conveying device; amovement amount detection unit that successively detects an amount ofmovement, by the conveying device, of the object supplied by the firstsupply unit onto the conveying device; a single vision sensor thatsuccessively acquires, on an upstream side of the robot in a conveyingdirection, visual information of the object being conveyed by theconveying device; an object detection unit that processes the visualinformation acquired by the vision sensor to detect position andorientation of the object; an interval detection unit that processes thevisual information acquired by the vision sensor to detect an intervalbetween objects on the conveying device in the conveying direction; acontrol unit that controls the robot based on the amount of movement ofthe object detected by the movement amount detection unit and theposition and orientation of the object detected by the object detectionunit; and a production management unit that causes the second supplyunit to supply the object at a position of the interval detected by theinterval detection unit, in a case where the interval is greater than apredetermined threshold.

According to the present aspect, while objects, which are successivelysupplied by the first supply unit, are being conveyed by the conveyingdevice, the amount of movement, by the conveying device, of an object issuccessively detected by the movement amount detection unit, and also,visual information is acquired by the vision sensor. When the acquiredvisual information is processed by the object detection unit, theposition and orientation of the object are detected, and also, when theacquired visual information is processed by the interval detection unit,an interval between objects in the conveying direction is detected.

The robot is controlled by the control unit based on the amount ofmovement and the position and orientation, and an object being conveyedby the conveying device is processed.

In this case, if the interval between objects detected by the intervaldetection unit is greater than a predetermined threshold, the productionmanagement unit causes the second supply unit to operate such that anobject is supplied at the position of the detected interval betweenobjects. Accordingly, reduction in productivity due to deficiency insupply of objects may be prevented, without placing, on the robot, aburden of picking up an object from a temporary placing table and whilepreventing an operation program from becoming complicated such that therobot can be made to determine whether or not to pick up an object fromthe temporary placing table.

In the aspect described above, the interval detection unit may output,to the production management unit, a signal to an effect that theinterval that is greater than the threshold is detected, in a case wherethe object is not detected in a predetermined object presence checkregion arranged in a field of view of the vision sensor, and theproduction management unit may, upon reception of the signal, cause thesecond supply unit to supply the object in the object presence checkregion.

An interval greater than the predetermined threshold may thus be easilydetected, and the production management unit may easily solve deficiencyin the supply of objects by causing the second supply unit to supply anobject at the interval position.

Furthermore, in the aspect described above, the vision sensor mayacquire the visual information at a frame rate of detecting the sameobject at least twice in the field of view, and the object presencecheck region may be arranged at a position on an upstream side away froma downstream end of the field of view at least by a distance allowingacquisition of the visual information by the vision sensor.

This allows the visual information to be acquired at least twice whilethe same object supplied by the first supply unit is being conveyed inone direction by the conveying device and is passing through the fieldof view of the vision sensor. Because an object which is supplied by thesecond supply unit in the object presence check region when there is noobject in the object presence check region at the time of firstacquisition of the visual information is moved to a downstream side bythe conveying device, second acquisition of the visual information isperformed between a downstream end of the object presence check regionand the downstream end of the field of view, and the position andorientation may be detected.

Moreover, in the aspect described above, the second supply unit may bearranged on a downstream side from a center of the object presence checkregion by the amount of movement from reception of a command from theproduction management unit to supply of the object onto the conveyingdevice.

Accordingly, in the case where absence of an object in the objectpresence check region is detected due to acquisition of the visualinformation by the vision sensor, and a command is output by theproduction management unit to cause the second supply unit to supply anobject, even if there is a time lag until an object is actually suppliedonto the conveying device, and even if an interval of only about thewidth of the object presence check region is present between objects, anobject may be more reliably supplied at a center position of theinterval between objects, because the second supply unit is arrangedtaking into account the time lag.

1. A robot system comprising: a conveying device that conveys an objectin one direction; a robot, installed near the conveying device, thatperforms a process on the object being conveyed by the conveying device;a first supply unit and a second supply unit that supply the object ontothe conveying device; a movement amount detection unit that successivelydetects an amount of movement, by the conveying device, of the objectsupplied by the first supply unit onto the conveying device; a singlevision sensor that successively acquires, on an upstream side of therobot in a conveying direction, visual information of the object beingconveyed by the conveying device; an object detection unit thatprocesses the visual information acquired by the vision sensor to detectposition and orientation of the object; an interval detection unit thatprocesses the visual information acquired by the vision sensor to detectan interval between objects on the conveying device in the conveyingdirection; a control unit that controls the robot based on the amount ofmovement of the object detected by the movement amount detection unitand the position and orientation of the object detected by the objectdetection unit; and a production management unit that causes the secondsupply unit to supply the object at a position of the interval detectedby the interval detection unit, in a case where the interval is greaterthan a predetermined threshold.
 2. The robot system according to claim1, wherein the interval detection unit outputs, to the productionmanagement unit, a signal to an effect that the interval that is greaterthan the threshold is detected, in a case where the object is notdetected in a predetermined object presence check region arranged in afield of view of the vision sensor, and the production management unit,upon reception of the signal, causes the second supply unit to supplythe object in the object presence check region.
 3. The robot systemaccording to claim 2, wherein the vision sensor acquires the visualinformation at a frame rate of detecting the same object at least twicein the field of view, and the object presence check region is arrangedat a position on an upstream side away from a downstream end of thefield of view at least by a distance allowing acquisition of the visualinformation by the vision sensor.
 4. The robot system according to claim2, wherein the second supply unit is arranged on a downstream side froma center of the object presence check region by the amount of movementfrom reception of a command from the production management unit tosupply of the object onto the conveying device.